U.S. Pat. No. 4,935,616 (further described in the Sandia Lab News, vol. 46, No. 19, Sep. 16, 1994) describes a scannerless range imaging system using either an amplitude-modulated high-power laser diode or an array of amplitude-modulated light emitting diodes (LEDs) to completely illuminate a target scene. An improved scannerless range imaging system that is capable of yielding color intensity images in addition to the 3D range images is described in commonly-assigned, U.S. Pat. No. 6,349,174 entitled “Method and Apparatus for a Color Scannerless Range Imaging System”. As used herein, a scannerless range imaging system will be referred to as a “SRI system”. In the formation of a three-dimensional image (which will herein refer to the combination of the intensity image and range image), the SRI system generates an “image bundle”, which includes both the intensity image and a collection of phase offset images which are used to construct the range image. This formation of a three-dimensional image by the SRI system is more fully described in the “Detailed Description of the Invention” section of this disclosure.
Correct formation of a three-dimensional image by the SRI system assumes the system is stable, or not moving, during the capture of the collection of phase offset images. A typical duration might be anywhere from 0.3 to 2.0 seconds or more (based on the speed of the burst mode of a typical digital camera, such as the Kodak DC290 Zoom Digital Camera). If the SRI system is mounted on a tripod or a monopod, or placed on top of or in contact with a stationary object, then the stability assumption is likely to hold. However, if the collection of phase offset images is captured while the SRI system is held in the hands of the photographer, the slightest jitter or movement of the hands may introduce stabilization errors that will adversely affect the formation of the three-dimensional image. For another example of unwanted motion, consider that the collection of phase offset images is captured by a film SRI system, such as the system described in commonly assigned U.S. Pat. No. 6,118,946, “Method and apparatus for scannerless range image capture using photographic film.” When the phase offset images are scanned, film positioning errors can contribute to an incorrect formation of the three-dimensional image.
The process of removing any unwanted motion from a sequence of images is called image stabilization. Some systems use optical, mechanical, or other physical means to correct for the unwanted motion at the time of capture or scanning. However, these systems are often complex and expensive. To provide stabilization for a generic digital image sequence, several digital image processing methods have been developed and described in the prior art.
A number of digital image processing methods use a specific camera motion model to estimate one or more parameters such as zoom, translation, rotation, etc. between successive frames in the sequences. These parameters are computed from a motion vector field that describes the correspondence between image points in two successive frames. The resulting parameters can then be filtered over a number of frames to provide smooth motion. An example of such a system can be found in a patent by Burt et al (U.S. Pat. No. 5,629,988). A fundamental assumption in these systems is that a global transformation dominates the motion between adjacent frames. In the presence of significant local motion, such as multiple objects moving with independent motion trajectories, these methods may fail due to the computation of erroneous global motion parameters. In addition, it may be difficult to apply these methods to a sequence of phase offset images because the intensity value at each pixel depends in part on the phase offsets used to capture each image. As a result of this dependence, sequential phase offset images may differ dramatically in overall intensity; only the information contained in the phase of the Fourier Transform of the image is similar.
Other digital image processing methods for removing unwanted motion make use of a technique known as phase correlation for precisely aligning successive frames. An example of such a method has been reported by Eroglu et al. (“A fast algorithm for subpixel accuracy image stabilization for digital film and video,” in Proc. SPIE Visual Communications and Image Processing, Vol. 3309, pp. 786-797, 1998). These methods would be more applicable to the stabilization of a sequence of phase offset images from an SRI camera than the aforementioned methods because the correlation procedure only compares the information contained in the phase of the Fourier Transform of the images. However, these methods only apply to sequences that have no local motion, or alternatively, a user must select a region in consecutive frames that has no local motion. The dependence upon areas with no local motion and the necessity for user intervention are major drawbacks of these methods.
One special feature of the SRI system is that only three phase offset images are required to form a three-dimensional image. However, more than three phase offset images can be used, and the use of more than three phase offset images will increase the accuracy of the three-dimensional image in general. All of the aforementioned digital image processing methods for removing unwanted motion from image sequences remove this motion in every frame of the image sequence. Therefore, an additional drawback to all of the aforementioned methods is that there exist no mechanisms for selectively removing or ignoring any frames where the unwanted motion is deemed severe. If the unwanted motion is too large or localized in any of the phase offset images, any errors introduced by the removal method degrade the resulting three-dimensional image to a quality lower than it would have been had the corresponding phase offset images been ignored.
Therefore, there exists a need in the art for a method of generating a stabilized three-dimensional image from phase offset images captured by a scannerless range imaging system, where the phase offset images contain unwanted motion; such a method should be capable of correcting unwanted motion if that motion is small, and of removing one or more phase offset images from the image bundle if that image motion is large or localized.